Hierarchical profiled scheduling and shaping

ABSTRACT

Various exemplary embodiments relate to a method and related network node including one or more of the following: determining, by the network node, that a port of the network node is ready to receive a packet; identifying a packet having a highest packet priority among a plurality of packets received via a plurality of interfaces, wherein the step of identifying includes, for each of a plurality of components at a first hierarchy level: identifying a first level highest priority packet among a plurality of packets available to the component, based on a packet priority associated with each of the plurality of packets available to the component, sharing the packet priority of the first level highest priority packet with at least one component at a second hierarchy level; and transmitting the packet having the highest priority to the port.

TECHNICAL FIELD

Various exemplary embodiments disclosed herein relate generally tonetwork routing.

BACKGROUND

Many hierarchical telecommunications networks, including the Internet,are centered around a core network. Numerous smaller networks attach tothe core network, thereby establishing communication with each other. Toenable communication between such subnetworks and the core network,backhaul devices are provided to aggregate subnetwork traffic fromvarious sources for transmission over the core network toward theirrespective destinations. As traffic exits the core network, similarbackhaul devices sort and transmit packets toward the appropriatesubnetworks for further routing.

Different traffic within such a telecommunications network often carriesdifferent priorities. For example, real-time video conferencing trafficmay carry a higher priority than best-effort traffic such as webbrowsing. Various network devices may process packets differentlydepending on this associated priority. For example, given the simplechoice between transmitting a real-time packet and a best-effort packet,a router may forward the real-time packet and either drop or delay thebest-effort packet.

SUMMARY

Various exemplary embodiments relate to a method performed by a networknode having a hierarchical organization for forwarding packets in acomputer network, the method including one or more of the following:determining, by the network node, that a port of the network node isready to receive a packet; identifying a packet having a highest packetpriority among a plurality of packets received via a plurality ofinterfaces, wherein the step of identifying includes, for each of aplurality of components at a first hierarchy level: identifying a firstlevel highest priority packet among a plurality of packets available tothe component at the first hierarchy level, based on a packet priorityassociated with each of the plurality of packets available to thecomponent at the first hierarchy level, sharing the packet priority ofthe first level highest priority packet with at least one component at asecond hierarchy level; and transmitting the packet having the highestpriority to the port.

Various exemplary embodiments relate to a network node for forwardingpackets, the network node including one or more of the following: atransmission port for transmitting packets; a plurality of interfacesfor receiving packets; a plurality of queues for storing packetsreceived over the plurality of interfaces, wherein a packet priority isassociated with each packet; and a plurality of first level shapers thatare each associated with a subset of the plurality of queues, whereineach first level shaper: identifies a packet having a first levelhighest packet priority of all packets available for transmission fromthe subset of the plurality of queues, and shares the first levelhighest packet priority with at least one other component; a secondlevel shaper that: receives an indication that the transmission port isready to receive a packet, receives at least one first level highestpacket priority shared by at least one of the plurality of first levelshapers; identifies an available packet associated a highest packetpriority based on the at least one first level highest packet priority,and transmits the available packet having the highest packet priority tothe transmission port.

Various exemplary embodiments relate to a machine-readable storagemedium encoded with instructions for execution by a network node havinga hierarchical organization for forwarding packets in a computernetwork, the machine-readable storage medium including one or more ofthe following: instructions for determining, by the network node, that aport of the network node is ready to receive a packet; instructions foridentifying a packet having a highest packet priority among a pluralityof packets received via a plurality of interfaces, wherein theinstructions for identifying include instructions for, for each of aplurality of components at a first hierarchy level: identifying a firstlevel highest priority packet among a plurality of packets available tothe component at the first hierarchy level, based on a packet priorityassociated with each of the plurality of packets available to thecomponent at the first hierarchy level, sharing the packet priority ofthe first level highest priority packet with at least one component at asecond hierarchy level; and instructions for transmitting the packethaving the highest priority over the port.

Various exemplary embodiments relate to a method performed by a packetprocessor having a hierarchical organization for forwarding packets in acomputer network, the method including one or more of the following:determining, by the packet processor, that the packet processor shouldtransmit a packet; identifying a packet having a highest packet priorityamong a plurality of packets received from a plurality of sources,wherein the step of identifying includes, for each of a plurality ofcomponents at a first hierarchy level: identifying a first level highestpriority packet among a plurality of packets available to the componentat the first hierarchy level, based on a packet priority associated witheach of the plurality of packets available to the component at the firsthierarchy level, sharing the packet priority of the first level highestpriority packet with at least one component at a second hierarchy level;and transmitting the packet having the highest priority.

Various exemplary embodiments relate to a packet processor forforwarding packets, the network node including one or more of thefollowing: a plurality of queues for storing received packets, wherein apacket priority is associated with each packet; and a plurality of firstlevel shapers that are each associated with a subset of the plurality ofqueues, wherein each first level shaper: identifies a packet having afirst level highest packet priority of all packets available fortransmission from the subset of the plurality of queues, and shares thefirst level highest packet priority with at least one other component; asecond level shaper that: determines that the packet processor shouldtransmit a packet, receives at least one first level highest packetpriority shared by at least one of the plurality of first level shapers;identifies an available packet associated a highest packet prioritybased on the at least one first level highest packet priority, andtransmits the available packet having the highest packet priority.

Various embodiments are described wherein a third hierarchy level islocated between the first hierarchy level and the second hierarchylevel.

Various embodiments are described wherein the plurality of packets is agroup of all packets identified as eligible for transmission by at leasta first plurality of traffic shaping components.

Various embodiments are described wherein each packet priority of theplurality of packets is determined based on at least one field carriedby the packet.

Various embodiments are described wherein the plurality of packets arestored among a plurality of queues. Such embodiments may additionallyinclude one or more of the following: determining a status for a queueholding a packet of the plurality of packets; determining a packetpriority for the packet based on the status of the queue.

Various embodiments are described wherein the status is one of committedinformation rate status and excess information rate status.

Various embodiments additionally include one or more of the following: athird level shaper associated with at least one first level shaper;wherein the first level shaper shares the first level highest packetpriority with at least one of the third level shaper and the secondlevel shaper.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to better understand various exemplary embodiments, referenceis made to the accompanying drawings, wherein:

FIG. 1 illustrates an exemplary network node for forwarding packets froma plurality of interfaces;

FIG. 2 illustrates an exemplary queue for storing packets;

FIG. 3 illustrates an exemplary level 2 shaper for transmitting packets;

FIG. 4 illustrates an exemplary level 1 shaper for transmitting packets;

FIG. 5 illustrates an exemplary level 0 shaper for transmitting packets;

FIG. 6 illustrates an exemplary method for transmitting a packet; and

FIG. 7 illustrates an exemplary method for determining a highestpriority packet.

DETAILED DESCRIPTION

Aggregation in backhaul devices introduces difficulties in respectingpacket priorities. For example, a backhaul device may implement ahierarchical system, wherein a component for each interface may be awareof the highest priority packet available. Next, a component for eachphysical card carrying a number of interfaces may be aware of a highestpriority interface on a card. Finally, a component associated with theoutgoing port may be aware of a highest priority card of all cards inthe device. This method, however, may lead to many results wherein thehighest priority packet across all interfaces must wait in queue for thetransmission of lower priority packets such as, for example, lowpriority packets arriving via a higher priority interface or card.Accordingly, there is a need for a hierarchical scheduling device thattruly schedules packets based on a packet priority.

Referring now to the drawings, in which like numerals refer to likecomponents or steps, there are disclosed broad aspects of variousexemplary embodiments.

FIG. 1 illustrates an exemplary network node 100 for forwarding packetsfrom a plurality of interfaces. As illustrated, node 100 may have ahierarchical structure with three hierarchical levels. Node 100 mayinclude a plurality of interfaces 110, 112, 114, 116; a plurality ofqueues 120, 121, 122, 123, 124, 125, 126, 127; a plurality of level 2shapers 130, 132, 134, 136; a plurality of level 1 shapers 140, 144;level 0 shaper 150; and port 0 160. It will be appreciated that node 100is in some respects a simplification and may include various additionalcomponents. For example, node 100 may include numerous additional ports(not shown). As another example, node 100 may include fewer oradditional hierarchy levels. Various additional modifications will beapparent to those of skill in the art.

It should be apparent that various components of node 100 may beimplemented as a single packet processor. For example, node 100 mayinclude a packet processor (not detailed) that, in turn, includesplurality of queues 120, 121, 122, 123, 124, 125, 126, 127; plurality oflevel 2 shapers 130, 132, 134, 136; plurality of level 1 shapers 140,144; and/or level 0 shaper 150 on a single chip or component.

Interfaces 110, 112, 114, 116 may each be an interface comprisinghardware and/or executable instructions encoded on a machine-readablestorage medium configured to receive messages from other nodes. Forexample, each interface 110, 112, 114, 116 may be an Ethernet portconnecting to a different network node. It should be understood that twoor more interfaces could connect to the same network node. Eachinterface 110, 112, 114, 116 may further be adapted to categorizereceived packets and store them in an appropriate queue accordingly.

Queues 120, 121, 122, 123, 124, 125, 126, 127 may each include hardwareand/or executable instructions on a machine-readable storage mediumconfigured to store various packets between time of receipt andtransmission. For example, each queue 120, 121, 122, 123, 124, 125, 126,127 may include a first-in-first-out (FIFO) buffer. Further, each queue120, 121, 122, 123, 124, 125, 126, 127 may be associated with aninterface 110, 112, 114, 116 and/or pre-configured quality of service(QoS) parameters. For example, queue 120 may be associated withguaranteed traffic received via interface 0 110 while queue 121 may beassociated with best-effort traffic received via interface 0 110.Alternatively or additionally, each queue 120, 121, 122, 123, 124, 125,126, 127 may be associated with a particular flow or group of flowsreceived via the associated interface 110, 112, 114, 116.

During operation, each queue 120, 121, 122, 123, 124, 125, 126, 127 mayoperate under different status. For example, each queue 120, 121, 122,123, 124, 125, 126, 127 may monitor the rate at which packets aretransferred from the queue 120, 121, 122, 123, 124, 125, 126, 127 anddetermine whether the queue 120, 121, 122, 123, 124, 125, 126, 127 iscurrently operating in a committed information rate (CIR) status or anexcess information rate (EIR) status. In this example, if the packets inqueue 120 are transferred at a rate higher than its configured CIR overa recent time window, queue 120 may switch to EIR status to indicatethat it is currently meeting its minimum throughput requirements and,accordingly, it may temporarily be treated as a lower priority.

Level 2 shapers 130, 132, 134, 136 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto determine which packet from the fronts of the associated queues 120,121, 122, 123, 124, 125, 126, 127 has the highest packet priority. Indoing so, level 2 shapers 130, 132, 134, 136 may take into accountcharacteristics and/or a current status of the queue 120, 121, 122, 123,124, 125, 126, 127. For example, level 2 shaper 130 may deem the packetat the front of queue 120 as the highest priority packet among thequeues 120, 121 because queue 120 is associated with real-time trafficand currently in a committed information rate status. As anotherexample, if queue 120 is instead in EIR status and queue 121 is in CIRstatus, level 2 shaper may instead deem a packet from queue 121 ashaving the highest packet priority, even though queue 121 storesbest-effort traffic.

After determining the highest priority packet available, level 2 shapers130, 132, 134, 136 may further indicate to other components the packetpriority of the highest priority packets. For example, level 2 shaper130 may determine that the packet at the front of queue 120 has thehighest packet priority, which is “1.” Level 2 shaper 130 may thenindicate to level 1 shaper 140 and level 0 shaper 150 that its highestpriority packet has a priority of “1.” Level 1 shaper 140 and level 0shaper 150 may then use this information to, in turn, determine thehighest priority packet available to those components.

Level 2 shaper 130, 132, 134, 136 may perform additional functions suchas, for example, packet transmission and rate shaping. Upon request fromanother component, such as level 1 shaper 140, 144, a level 2 shaper130, 132, 134, 136 may transmit the packet having the highest priorityfrom queue 120, 121, 122, 123, 124, 125, 126, 127. If more than onepacket is associated with the highest priority packet, level 2 shaper130, 132, 134, 136 may use some method for determining which of thepackets to transmit such as, for example, the round robin algorithm. Aslevel 2 shaper 130, 132, 134, 136 transmits packets, it may monitor therate at which packets are being transmitted. If level 2 shaper 130, 132,134, 136 determines that it is transmitting packets at a rate that ishigher than a preconfigured maximum rate, level 2 shaper 130, 132, 134,136 may indicate to at least one other component that it is temporarilyineligible to transmit a packet.

Level 1 shapers 140, 144 may each include hardware and/or executableinstructions on a machine-readable storage medium configured todetermine which associated level 2 shaper 130, 132, 134, 136 has accessto the highest priority packet available to the level 1 shaper 140, 144.As previously described, each level 2 shaper 130, 132, 134, 136 mayindicate the highest available packet priority to the associated level 1shaper 140, 144. Level 1 shaper 140, 144 may use this information todetermine the highest packet priority available to that component. Forexample, level 2 shaper 130 may indicate to level 1 shaper 140 that ithas access to a packet of priority “1” while level 2 shaper 132 mayindicate that it has access to a packet of priority “2.” Accordingly,level 1 shaper 140 may determine that level 2 shaper 130 currentlyprovides the highest priority packet.

Level 1 shapers 140, 144 may perform additional functions such as, forexample, packet transmission and rate shaping. Upon request from anothercomponent, such as level 0 shaper 150, a level 1 shaper 140, 144 mayrequest the packet having the highest priority from level 2 shaper 130,132, 134, 136. If more than one packet is associated with the highestpriority packet, level 1 shaper 140, 144 may use some method fordetermining which of the packets to transmit such as, for example, theround robin algorithm. As level 1 shaper 140, 144 transmits packets, itmay monitor the rate at which packets are being transmitted. If 1 level1 shaper 140, 144 determines that it is transmitting packets at too higha rate, level 1 shaper 140, 144 may indicate to at least one othercomponent that it is temporarily ineligible to transmit a packet.

Level 0 shaper 150 may include hardware and/or executable instructionson a machine-readable storage medium configured to determine whichassociated level 1 shaper 140, 144 has access to the highest prioritypacket available to the level 0 shaper 150. As previously described,each level 2 shaper 130, 132, 134, 136 may indicate the highestavailable packet priority to the associated level 0 shaper 150. Level 0shaper 150 may use this information to determine the highest packetpriority available to that component. For example, level 2 shaper 130may indicate to level 0 shaper 140 that it has access to a packet ofpriority “1;” level 2 shaper 132 may indicate that it has access to apacket of priority “2;” level 2 shaper 134 may indicate that it hasaccess to a packet of priority “3;” and level 2 shaper 136 may indicatethat it has access to a packet of priority “2.” Accordingly, level 0shaper 150 may determine that level 2 shaper 130, and consequently level1 shaper 140, currently provides the highest priority packet.

Level 0 shaper 150 may provide additional functions such as portmonitoring and packet transmission. Level 0 shaper 150 may monitor port160 to determine when port 0 160 is ready to transmit the next packet.For example, port 0 160 may transmit a signal to level 0 shaper 150 orlevel 0 shaper 150 may monitor the fill level of an outgoing queue ofport 0 160. Regardless of the method used, once level 0 shaper 150determines that port 0 160 is ready for the next packet, it may transmita request for a packet to the level 1 shaper 140, 144 that has access tothe highest priority packet. In the case where multiple level 2 shapers140, 144 have access to a packet of highest priority, level 0 shaper 150may use some method for determining which of the packets to transmitsuch as, for example, the round robin algorithm.

Port 0 160 may be an interface comprising hardware and/or executableinstructions encoded on a machine-readable storage medium configured totransmit messages to other nodes. For example, port 0 may be an Ethernetport connecting to a different network node such as, for example, a corenetwork device.

Having described the components of network node 100, a brief summary ofthe operation of network node 100 will be provided. It should beapparent that the following description is intended to provide anoverview of the operation of network node 100 and is therefore asimplification in some respects. The detailed operation of network node100 will be described in further detail below in connection with FIGS.2-7. Note that while the following example assumes the presence of onlythose components illustrates in FIG. 1, additional and/or duplicatecomponents may be included in a network node.

Level 2 shaper 130 may presently indicate to level 1 shaper 140 andlevel 0 shaper 150 that its highest priority packet has a priority of“1.” Likewise, level 2 shapers 132, 134, 136 may indicate that theirhighest priority packets carry priorities of “2,” “3,” and “2,”respectively. When level 0 shaper 150 determines that port 0 160 isready for another packet, level 0 shaper 150 may request a packet fromlevel 1 shaper 140, because it offers the highest priority packet. Level1 shaper 140, in turn, may request a packet from level 2 shaper 130because level 2 shaper 130 offers the highest priority packet. Level 2shaper may dequeue the packet from queue 120 and transmit the packet tolevel 1 shaper 140 which, in turn, transmits the packet to level 0shaper 150. Level 0 shaper 150 then transmits the packet to port 0 160.

After transmitting a packet, level 2 shaper 130 may determine that itsnew highest priority packet has a priority of “2.” Level 2 shaper 130may further determine that it is currently exceeding its maximum rateand may indicate to level 1 shaper 140 and/or level 0 shaper 150 that itis currently ineligible to transmit a packet. Accordingly, for the nextpacket, level 0 shaper 150 may use round robin to determine whether totransmit the packet offered by level 2 shaper 132 or the packet offeredby level 2 shaper 136, because both packets carry the highest availablepriority of “2.”

FIG. 2 illustrates an exemplary queue 200 for storing packets. Queue 200may correspond to one or more of queues 120, 121, 122, 123, 124, 125,126, 127. Queue 200 may include a buffer 210, attribute reporter 220,activity monitor 230, and status reporter 240.

Buffer 210 may be any machine-readable medium capable of storingpackets. Accordingly, buffer may include a machine-readable storagemedium such as random-access memory (RAM), magnetic disk storage media,optical storage media, flash-memory devices, and/or similar storagemedia. Buffer 210 may store incoming packets and transmit packets in afirst-in-first-out (FIFO) order. Such incoming packets may arrive via aninterface, such as one or more of interfaces 110, 112, 114, 116.

Attribute reporter 220 may include hardware and/or executableinstructions on a machine-readable storage medium configured to storeconfiguration information regarding the type of packets to be stored inqueue 200. Attribute reporter 220 may further be configured to reportsuch configuration information to other components such as, for example,one or more of interfaces 110, 112, 114, 116, and/or one or more oflevel 2 shapers 130, 132, 134, 136. For example, attribute reporter 220may indicate that packets having a class of service (CoS) tag of “6” or“7” should be stored in queue 200. It should be apparent that otherattributes may be used other than CoS to determine which packets shouldbe stored in queue 200. Alternatively or additionally, attributereporter 220 may indicate particular flows which should be stored inqueue 200. Attribute reporter 220 may be preconfigured and/or configuredduring runtime.

Activity monitor 230 may include hardware and/or executable instructionson a machine-readable storage medium configured to monitor buffer 210and estimate or otherwise determine a current output rate of packetsfrom buffer 210. For example, activity monitor 230 may maintain acounter that is incremented each time a packet is transmitted frombuffer 210. This counter may be reduced or cleared periodically, suchthat it serves as an estimate of the current transmission rate for queue200. Other methods of estimating or determining a current activity ratefor queue 200 will be apparent to those of skill in the art.

Status reporter 240 may include hardware and/or executable instructionson a machine-readable storage medium configured to determine a currentstatus for queue 200. For example, in various embodiments, queue 200 mayoperate in CIR or EIR status. Using the current activity leveldetermined by activity monitor 230, status reporter may determine whichunder status queue 200 is currently operating. For example, if thecurrent activity is greater than a predetermined threshold, statusreporter 240 may indicate to other components that queue 200 iscurrently in EIR status. Otherwise, status reporter 240 may indicate toother components that queue 200 is currently in CIR status. Statusreporter 240 may report this status information to one or more othercomponents such as, for example, one or more of level 2 shapers 130,132, 134, 136.

FIG. 3 illustrates an exemplary level 2 shaper 300 for transmittingpackets. Level 2 shaper 300 may correspond to one or more of level 2shapers 130, 132, 134, 136. Level 2 shaper 300 may include transmitsignal monitor 310, packet transmitter 320, queue interfaces 330, 331,queue status and attribute interfaces 340, 341, highest priorityreporter 350, highest priority calculator 360, priority calculator 370,rate shaper 380, and eligibility reporter 390.

Transmit signal monitor 310 may include hardware and/or executableinstructions on a machine-readable storage medium configured to receivea signal from another component indicating a request to transmit apacket. For example, one or more of level 1 shapers 140, 144 may send asignal to transmit signal monitor 310 requesting transmission of apacket. In response, transmit signal monitor 310 may instruct packettransmitter 320 to transmit a packet. Transmit signal monitor 310 mayalso inform rate shaper 380 that a packet has been transmitted.

Packet transmitter 320 may include hardware and/or executableinstructions on a machine-readable storage medium configured to requestand receive a packet via one or more of queue interfaces 330, 331. Uponreceiving a requested packet, packet transmitter 320 may forward thepacket to the requesting component such as, for example, one or more oflevel 1 shapers 140, 144. Upon receiving an indication from transmitsignal monitor 310 that a packet should be transmitted, packettransmitter 320 may determine which queue holds the highest prioritypacket. For example, as will be described in further detail with respectto highest priority reporter 350, highest priority reporter 350 mayindicate one or more of queue interfaces 330, 331 from which packettransmitter 320 may obtain a packet carrying the highest availablepriority. If more than one of interfaces 330, 331 may provide a highestpriority packet, packet transmitter 320 may use some method, such asround robin, to select a queue interface 330, 331 from which to requesta packet.

Queue interfaces 330, 331 may each include hardware and/or executableinstructions on a machine-readable storage medium configured tocommunicate with queues such as, for example, one or more of queues 120,121, 122, 123, 124, 125, 126, 127. Accordingly, queue interfaces 330,331 may be operable to transmit requests for packets and/or receivepacket data in response.

Queue status and attribute interfaces 340, 341 may each include hardwareand/or executable instructions on a machine-readable storage mediumconfigured to communicate with queues such as, for example, one or moreof queues 120, 121, 122, 123, 124, 125, 126, 127. Accordingly, queueinterfaces 330, 331 may be operable to transmit requests for data and/orreceive status and/or queue attribute information.

Highest priority reporter 350 may include hardware and/or executableinstructions on a machine-readable storage medium configured to transmita signal to one or more components indicating the highest priority amongthe packets available to shaper 300. For example, highest priorityreporter may send the highest priority value to one or more of level 1shapers 140, 144, and/or level 0 shaper 150. Highest priority reporter350 may have knowledge of the highest priority available from highestpriority calculator. Highest priority reporter may additionally indicateto packet transmitter 320 which interfaces and/or queues currently offera packet associated with the highest priority.

Highest priority calculator 360 may include hardware and/or executableinstructions on a machine-readable storage medium configured todetermine the highest priority among all packets currently availablefrom a queue associated with shaper 300. Highest priority calculator 360may receive a current priority for each queue from priority calculator370 and identify the highest value of the priorities reported. Highestpriority calculator 360 may further be adapted to indicate which queuescurrently offer a packet having the highest priority to highest priorityreporter 350 and/or packet transmitter 320.

Priority calculator 370 may include hardware and/or executableinstructions on a machine-readable storage medium configured todetermine, for each queue associated with shaper 300, the priority ofthe currently offered packet. For example, in an embodiment wherein eachqueue is configured to correspond to a CoS and reports one of a CIR andan EIR status, priority calculator 370 may retrieve this information viaone or more of queue status and attribute interfaces 340, 341. Prioritycalculator 370 may then use the status and attribute information todetermine a current priority for the queue. In various embodiments,priority calculator 370 may assign one of four priorities to a queue.For example, priority calculator 370 may determine that a queueassociated with a CoS of “5” or higher and operating in CIR status has acurrent priority of “1;” a queue associated with a CoS of “4” or lowerand operating in CIR status has a current priority of “2;” a queueassociated with a CoS of “5” or higher and operating in EIR status has acurrent priority of “3;” and a queue associated with a CoS of “4” orlower and operating in EIR status has a current priority of “4.” Itshould be apparent that a number of different methods to assign prioritymay be used and that additional or alternative information may be usedin determining a priority value. Further, greater or fewer than fourpriority levels may be assigned.

Rate shaper 380 may include hardware and/or executable instructions on amachine-readable storage medium configured to monitor and/or estimate acurrent rate of transmission for shaper 300. For example, rate shaper380 may include a counter that is incremented each time transmit signalmonitor 310 indicates that a packet is being transmitted. Rate shaper380 may periodically decrease or reset this counter in order to maintaina current estimation of the transmission rate. Other methods fordetermining or estimating a recent transmission rate will be apparent tothose of skill in the art. Rate shaper 380 may further be configuredwith a threshold for rate shaping. For example, if the currenttransmission rate exceeds the configured threshold (such as, forexample, 1 mbps), rate shaper 380 may indicate to eligibility reporter390 that packet transmission should be temporarily halted.

Eligibility reporter 390 may include hardware and/or executableinstructions on a machine-readable storage medium configured to indicateto one or more other components whether shaper 300 is currently eligibleto transmit a packet. For example, eligibility reporter 390 maycommunicate this information to one or more of level 1 shapers 140, 144,and/or level 0 shaper 150.

In various alternative embodiments, eligibility and highest availablepriority may not be separately reported. For example, if shaper 300determines that it is currently ineligible to transmit a packet, shaper300 may simply indicate the lowest possible priority as the highestavailable priority packet to discourage requests for packettransmission. In such embodiments, highest priority reporter 350 andeligibility reporter 390 may be the same component. Various additionalmodifications will be apparent to those of skill in the art.

FIG. 4 illustrates an exemplary level 1 shaper 400 for transmittingpackets. Level 1 shaper 400 may correspond to one or more of level 1shapers 140, 144. Level 1 shaper 400 may include transmit signal monitor410, packet transmitter 420, L2 interfaces 430, 432, L2 priorityinterfaces 440, 442, L2 eligibility interfaces 450, 452 highest eligiblepriority calculator 460, rate shaper 480, and eligibility reporter 490.

Transmit signal monitor 410 may include hardware and/or executableinstructions on a machine-readable storage medium configured to receivea signal from another component indicating a request to transmit apacket. For example, level 0 shapers 150 may send a signal to transmitsignal monitor 410 requesting transmission of a packet. In response,transmit signal monitor 410 may instruct packet transmitter 420 totransmit a packet. Transmit signal monitor 410 may also inform rateshaper 480 that a packet has been transmitted.

Packet transmitter 420 may include hardware and/or executableinstructions on a machine-readable storage medium configured to requestand receive a packet via one or more of L2 interfaces 430, 432. Uponreceiving a requested packet, packet transmitter 420 may forward thepacket to the requesting component such as, for example, level 0 shapers150. Upon receiving an indication from transmit signal monitor 410 thata packet should be transmitted, packet transmitter 420 may determinewhich L2 shaper offers the highest priority packet. For example, as willbe described in further detail with respect to highest eligible prioritycalculator 460, highest eligible priority calculator 460 may indicateone or more of L2 interfaces 430, 432 from which packet transmitter 420may obtain a packet carrying the highest available priority. If morethan one of L2 interfaces 430, 432 may provide a highest prioritypacket, packet transmitter 420 may use some method, such as round robin,to select a L2 interface 430, 432 from which to request a packet.

L2 interfaces 430, 432 may each include hardware and/or executableinstructions on a machine-readable storage medium configured tocommunicate with L2 components such as, for example, one or more of L2shapers 130, 132, 134, 136. Accordingly, L2 interfaces 430, 432 may beoperable to transmit requests for packets and/or receive packet data inresponse.

L2 priority interfaces 440, 442 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto communicate with L2 components such as, for example, one or more ofL2 shapers 130, 132, 134, 136. Accordingly, L2 priority interfaces 440,442 may be operable to receive a highest available priority value fromeach such L2 component. Such highest available priority may betransmitted, for example, by a component such as highest priorityreporter 350 of shaper 300.

L2 eligibility interfaces 450, 452 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto communicate with L2 components such as, for example, one or more ofL2 shapers 130, 132, 134, 136. Accordingly, L2 eligibility interfaces450, 452 may be operable to receive an indication from each such L2component as to whether the L2 component is currently eligible totransmit a packet. Such indication may be transmitted, for example, by acomponent such as eligibility reporter 390 of shaper 300.

Highest eligible priority calculator 460 may include hardware and/orexecutable instructions on a machine-readable storage medium configuredto determine which L2 components currently offer a packet with thehighest priority. Accordingly, for each L2 component associated withshaper 400, highest eligible priority calculator 460 may retrievepriority information via L2 priority interfaces 440, 442 and eligibilityinformation via L2 eligibility interfaces 450, 452. Highest eligiblepriority calculator 460 may then determine, for those associated L2devices currently eligible to transmit a packet, the highest prioritypacket available. Highest eligible priority calculator 460 may thenindicate to packet transmitter 420 which L2 components offer a packethaving the highest eligible priority. In various alternative embodimentswherein eligibility is not separately reported, highest eligiblepriority calculator 460 may simply determine the highest reportedpriority and L2 eligibility interfaces 450, 452 may not be present.

Rate shaper 480 may include hardware and/or executable instructions on amachine-readable storage medium configured to monitor and/or estimate acurrent rate of transmission for shaper 400. For example, rate shaper480 may include a counter that is incremented each time transmit signalmonitor 410 indicates that a packet is being transmitted. Rate shaper480 may periodically decrease or reset this counter in order to maintaina current estimation of the transmission rate. Other methods fordetermining or estimating a recent transmission rate will be apparent tothose of skill in the art. Rate shaper 480 may further be configuredwith a threshold for rate shaping. For example, if the currenttransmission rate exceeds the configured threshold (such as, forexample, 1 mbps), rate shaper 480 may indicate to eligibility reporter490 that packet transmission should be temporarily halted.

Eligibility reporter 490 may include hardware and/or executableinstructions on a machine-readable storage medium configured to indicateto one or more other components whether shaper 400 is currently eligibleto transmit a packet. For example, eligibility reporter 490 maycommunicate this information to level 0 shaper 150.

In various alternative embodiments wherein the level 2 shapers providean available highest priority to level 1 shapers but not level 0shapers, level 1 shaper 400 may additionally include a highest priorityreporter (not shown), similar to highest priority reporter 350 of level2 shaper 300. As such, each level 1 shaper may report the highestpriority available to that component. The provided L0 shaper may thenreceive a smaller number of such highest available priorities.Additional modifications useful in implementing such alternativeembodiments will be apparent to those of skill in the art.

FIG. 5 illustrates an exemplary level 0 shaper 500 for transmittingpackets. Level 0 shaper 500 may correspond to level 0 shaper 150. Level0 shaper 500 may include port monitor 510, packet transmitter 520, L1interfaces 530, 534, L2 priority interfaces 540, 546, L2 eligibilityinterfaces 550, 556, L1 eligibility interfaces 560, 564, and highesteligible priority calculator 570.

Port monitor 510 may include hardware and/or executable instructions ona machine-readable storage medium configured to monitor a port, such asport 0 160, and/or receive a signal from such a port indicating that theport is ready to receive a packet. In response, port monitor 510 mayinstruct packet transmitter 520 to transmit a packet.

Packet transmitter 520 may include hardware and/or executableinstructions on a machine-readable storage medium configured to requestand receive a packet via one or more of L1 interfaces 530, 534. Uponreceiving a requested packet, packet transmitter 520 may forward thepacket to the ready port such as, for example, port 0 160. Uponreceiving an indication from port monitor 510 that a packet should betransmitted, packet transmitter 520 may determine which L1 componentcurrently offers the highest priority packet. For example, as will bedescribed in further detail with respect to highest eligible prioritycalculator 570, highest eligible priority calculator 570 may indicateone or more of L1 interfaces 530, 534 from which packet transmitter 520may obtain a packet carrying the highest available priority. If morethan one of interfaces 530, 534 may provide a highest priority packet,packet transmitter 520 may use some method, such as round robin, toselect a queue interface 530, 534 from which to request a packet.

L1 interfaces 530, 534 may each include hardware and/or executableinstructions on a machine-readable storage medium configured tocommunicate with L1 components such as, for example, one or more of L1shapers 140, 144. Accordingly, L1 interfaces 530, 534 may be operable totransmit requests for packets and/or receive packet data in response.

L2 priority interfaces 540, 546 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto communicate with L2 components such as, for example, one or more ofL2 shapers 130, 132, 134, 136. Accordingly, L2 priority interfaces 540,546 may be operable to receive a highest available priority value fromeach such L2 component. Such highest available priority may betransmitted, for example, by a component such as highest priorityreporter 350 of shaper 300.

In various alternative embodiments wherein the level 2 shapers providean available highest priority to level 1 shapers but not level 0shapers, level 1 shapers may report their highest priorities to thelevel 0 shapers. In such embodiments, level 0 shaper 500 mayalternatively or additionally include a number of L1 priority interfaces(not shown) that operate in a similar manner to L2 priority interfaces,instead communicating with a number of L1 components. Additionalmodifications useful in implementing such alternative embodiments willbe apparent to those of skill in the art.

L2 eligibility interfaces 550, 556 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto communicate with L2 components such as, for example, one or more ofL2 shapers 130, 132, 134, 136. Accordingly, L2 eligibility interfaces550, 556 may be operable to receive an indication from each such L2component as to whether the L2 component is currently eligible totransmit a packet. Such indication may be transmitted, for example, by acomponent such as eligibility reporter 390 of shaper 300.

L1 eligibility interfaces 560, 564 may each include hardware and/orexecutable instructions on a machine-readable storage medium configuredto communicate with L1 components such as, for example, one or more ofL1 shapers 140, 144. Accordingly, L1 eligibility interfaces 560, 564 maybe operable to receive an indication from each such L1 component as towhether the L1 component is currently eligible to transmit a packet.Such indication may be transmitted, for example, by a component such aseligibility reporter 490 of shaper 400.

Highest eligible priority calculator 570 may include hardware and/orexecutable instructions on a machine-readable storage medium configuredto determine which L1 components currently offer a packet with thehighest priority. Accordingly, for each L2 component associated withshaper 500, highest eligible priority calculator 570 may retrievepriority information via L2 priority interfaces 540, 546 and eligibilityinformation via L2 eligibility interfaces 550, 556 and/or L1 eligibilityinterfaces 560, 564. Highest eligible priority calculator 570 may thendetermine, for those associated L1 devices currently eligible totransmit a packet, the highest priority packet available. Highesteligible priority calculator 570 may then indicate to packet transmitter520 which L1 components offer a packet having the highest eligiblepriority. In various alternative embodiments wherein eligibility is notseparately reported, highest eligible priority calculator 570 may simplydetermine the highest reported priority and L2 eligibility interfaces550, 556 and/or L1 eligibility interfaces 560, 564 may not be present.

FIG. 6 illustrates an exemplary method 600 for transmitting a packet.Exemplary method 600 may be performed, for example, by the components ofnetwork node 100. Method 600 may begin in step 605 and proceed to step610, where network node 100 may determine that an outgoing port is readyto receive a packet. For example, port 0 160 may transmit a signal tolevel 0 shaper 150.

Next, in step 620, network node 100 may determine the highest prioritypacket available for transmission across all incoming interfaces. Forexample, each L2 component 130, 132, 134, 136 may calculate a highestavailable priority and send the value to level 0 shaper 150. Level 0shaper 150 may then select the highest priority of those prioritiesreported for transmission. In various embodiments wherein variouscomponents perform rate shaping and report eligibility, as described indetail above, node 100 may only select a packet for transmission fromthose packets that are eligible for transmission. Thus, the packettransmitted may be a packet having the highest priority of the set ofeligible packets across all interfaces.

Once the highest priority packet is identified, node 100 transmits thepacket to the readied port in step 630. In various embodiments, step 630may include enqueueing the packet on an outgoing buffer of the portand/or presently transmitting the packet via the port to another networknode. Method 600 may then proceed to end in step 635.

FIG. 7 illustrates an exemplary method 700 for determining a highestpriority packet. Method 700 may correspond to step 620 of method 600.Method 700 may begin in step 705 and proceed to step 710 where each L2component determines the highest priority among the packets available tothat component. Each L2 component may also report this information tothe appropriate L1 and L0 components of node 100. Next, in step 720,each L1 component may use the priorities reported by the associated L2components to determine which L2 components offer packets carrying thehighest available priority. Similarly, in step 730, each L0 componentmay use the priorities reported by the associated L2 components todetermine which L1 components offer packets carrying the highestavailable priority. Method 700 may then end in step 735.

According to the foregoing, various exemplary embodiments provide anetwork node that more fully accounts for packet priorities whenscheduling packet transmissions. In particular, by sharing a packetpriority at all levels of a shaping and/or scheduling hierarchy, eachcomponent within the node has access to information helpful in selectingan optimum packet for transmission.

It should be apparent from the foregoing description that variousexemplary embodiments of the invention may be implemented in hardwareand/or firmware. Furthermore, various exemplary embodiments may beimplemented as instructions stored on a machine-readable storage medium,which may be read and executed by at least one processor to perform theoperations described in detail herein. A machine-readable storage mediummay include any mechanism for storing information in a form readable bya machine, such as a personal or laptop computer, a server, or othercomputing device. Thus, a machine-readable storage medium may includeread-only memory (ROM), random-access memory (RAM), magnetic diskstorage media, optical storage media, flash-memory devices, and similarstorage media.

It should be appreciated by those skilled in the art that any blockdiagrams herein represent conceptual views of illustrative circuitryembodying the principles of the invention. Similarly, it will beappreciated that any flow charts, flow diagrams, state transitiondiagrams, pseudo code, and the like represent various processes whichmay be substantially represented in machine readable media and soexecuted by a computer or processor, whether or not such computer orprocessor is explicitly shown.

Although the various exemplary embodiments have been described in detailwith particular reference to certain exemplary aspects thereof, itshould be understood that the invention is capable of other embodimentsand its details are capable of modifications in various obviousrespects. As is readily apparent to those skilled in the art, variationsand modifications can be effected while remaining within the spirit andscope of the invention. Accordingly, the foregoing disclosure,description, and figures are for illustrative purposes only and do notin any way limit the invention, which is defined only by the claims.

What is claimed is:
 1. A method performed by a network node having atransmission port, plurality of queues, a plurality of first levelshapers, a second level shaper, and a third level shaper for forwardingpackets in a computer network, the method comprising: storing aplurality of packets in the plurality of queues, wherein a packetpriority is associated with each packet of the plurality of packets;indicating by each queue of the plurality of queues that a packet at thefront of the queue is available for transmission; identifying, at afirst level shaper of the plurality of first level shapers, a packethaving a first level highest packet priority of all packets availablefor transmission; sharing, at the first level shaper, the highest packetpriority with at least one other component; receiving, at the secondlevel shaper, an indication that the transmission port is ready toreceive a packet; receiving, at the second level shaper, at least onefirst level highest packet priority shared by at least one of theplurality of first level shapers; identifying, at the second levelshaper, an available packet associated with a highest packet prioritybased on the at least one first level highest packet priority;transmitting, at the second level shaper, the available packet havingthe highest packet priority to the transmission port; monitoring, at thethird level shaper, an activity level of the third level shaper;determining, at the third level shaper, whether the third level shaperis eligible to transmit a packet based on the activity level; andindicating, at the third level shaper, when the third level shaper isineligible to transmit a packet, that no packet is available fortransmission.
 2. The method of claim 1, wherein the plurality of packetsis a group of all packets identified as eligible for transmission by atleast a first plurality of traffic shaping components.
 3. The method ofclaim 1, wherein each packet priority of the plurality of packets isdetermined based on at least one field carried by the packet.
 4. Themethod of claim 1, wherein the plurality of packets are stored among aplurality of queues, the method further comprising: determining a statusfor a queue holding a packet of the plurality of packets; determining apacket priority for the packet based on the status of the queue.
 5. Themethod of claim 4, wherein the status is one of committed informationrate status and excess information rate status.
 6. A network node forforwarding packets, the network node comprising: a transmission port fortransmitting packets; a plurality of interfaces for receiving packets; aplurality of queues for storing packets received over the plurality ofinterfaces, wherein a packet priority is associated with each packet andeach queue of the plurality of queues indicates that a packet at thefront of the queue is available for transmission; a plurality of firstlevel shapers that are each associated with a subset of the plurality ofqueues, wherein each first level shaper: identifies a packet having afirst level highest packet priority of all packets available fortransmission from the subset of the plurality of queues, and shares thefirst level highest packet priority with at least one other component; asecond level shaper that: receives an indication that the transmissionport is ready to receive a packet, receives at least one first levelhighest packet priority shared by at least one of the plurality of firstlevel shapers; identifies an available packet associated a highestpacket priority based on the at least one first level highest packetpriority, and transmits the available packet having the highest packetpriority to the transmission port; and a plurality of third levelshapers between the plurality of first level shapers and the secondlevel shaper that each comprise a rate shaper that: monitors an activitylevel for the third level shaper, determines whether the third levelshaper is eligible to transmit a packet based on the activity level, andif the third level shaper is ineligible to transmit a packet, indicatesto at the second level shaper that no packet is available fortransmission.
 7. The network node of claim 6, wherein the availablepacket associated with the highest packet priority is selected from agroup of packets identified as eligible for transmission by at least afirst plurality of shapers.
 8. The network node of claim 6, wherein eachpacket priority is determined based on at least one field carried by theassociated packet.
 9. The network node of claim 6, wherein each queue ofthe plurality of queues comprises an activity monitor that determines astatus for the queue based on observed activity; and the packet priorityfor each packet is based on the status of the queue that stores thepacket.
 10. The network node of claim 9, wherein the status is one ofcommitted information rate status and excess information rate status.11. The network node of claim 6, further comprising: a third levelshaper associated with at least one first level shaper; wherein thefirst level shaper shares the first level highest packet priority withat least one of the third level shaper and the second level shaper. 12.A non-transitory machine-readable storage medium encoded withinstructions for execution by a network node having a transmission port,plurality of queues, a plurality of first level shapers, a second levelshaper, and a third level shaper for forwarding packets in a computernetwork, the machine-readable storage medium comprising: instructionsfor storing a plurality of packets in the plurality of queues, wherein apacket priority is associated with each packet of the plurality ofpackets; instructions for indicating by each queue of the plurality ofqueues that a packet at the front of the queue is available fortransmission; instructions for identifying, at a first level shaper ofthe plurality of first level shapers, a packet having a first levelhighest packet priority of all packets available for transmission;instructions for sharing, at the first level shaper, the highest packetpriority with at least one other component; instructions for receiving,at the second level shaper, an indication that the transmission port isready to receive a packet; instructions for receiving, at the secondlevel shaper, at least one first level highest packet priority shared byat least one of the plurality of first level shapers; instructions foridentifying, at the second level shaper, an available packet associatedwith a highest packet priority based on the at least one first levelhighest packet priority; instructions for transmitting, at the secondlevel shaper, the available packet having the highest packet priority tothe transmission port; instructions for monitoring, at the third levelshaper, an activity level of the third level shaper; instructions fordetermining, at the third level shaper, whether the third level shaperis eligible to transmit a packet based on the activity level; andinstructions for indicating, at the third level shaper, when the thirdlevel shaper is ineligible to transmit a packet, that no packet isavailable for transmission.
 13. The non-transitory machine-readablestorage medium of claim 12, wherein the plurality of packets is a groupof all packets identified as eligible for transmission by at least afirst plurality of traffic shaping components.
 14. The non-transitorymachine-readable storage medium of claim 12, wherein each packetpriority of the plurality of packets is determined based on at least onefield carried by the packet.
 15. The non-transitory machine-readablestorage medium of claim 12, wherein the plurality of packets are storedamong a plurality of queues, the machine-readable storage medium furthercomprising: instructions for determining a status for a queue holding apacket of the plurality of packets; instructions for determining apacket priority for the packet based on the status of the queue.
 16. Thenon-transitory machine-readable storage medium of claim 12, wherein thestatus is one of committed information rate status and excessinformation rate status.
 17. A method performed by a packet processorhaving a transmission port, plurality of queues, a plurality of firstlevel shapers, a second level shaper, and a third level shaper forforwarding packets in a computer network, the method comprising: storinga plurality of packets in the plurality of queues, wherein a packetpriority is associated with each packet of the plurality of packets;indicating by each queue of the plurality of queues that a packet at thefront of the queue is available for transmission; identifying, at afirst level shaper of the plurality of first level shapers, a packethaving a first level highest packet priority of all packets availablefor transmission; sharing, at the first level shaper, the highest packetpriority with at least one other component; receiving, at the secondlevel shaper, an indication that the transmission port is ready toreceive a packet; receiving, at the second level shaper, at least onefirst level highest packet priority shared by at least one of theplurality of first level shapers; identifying, at the second levelshaper, an available packet associated with a highest packet prioritybased on the at least one first level highest packet priority;transmitting, at the second level shaper, the available packet havingthe highest packet priority to the transmission port; monitoring, at thethird level shaper, an activity level of the third level shaper;determining, at the third level shaper, whether the third level shaperis eligible to transmit a packet based on the activity level; andindicating, at the third level shaper, when the third level shaper isineligible to transmit a packet, that no packet is available fortransmission.
 18. The method of claim 17, wherein the plurality ofpackets is a group of all packets identified as eligible fortransmission by at least a first plurality of traffic shapingcomponents.
 19. The method of claim 17, wherein each packet priority ofthe plurality of packets is determined based on at least one fieldcarried by the packet.
 20. The method of claim 17, wherein the pluralityof packets are stored among a plurality of queues, the method furthercomprising: determining a status for a queue holding a packet of theplurality of packets; determining a packet priority for the packet basedon the status of the queue.
 21. The method of claim 20, wherein thestatus is one of committed information rate status and excessinformation rate status.
 22. A packet processor for forwarding packets,the packet processor comprising: a plurality of queues for storingreceived packets, wherein a packet priority is associated with eachpacket and each queue of the plurality of queues indicates that a packetat the front of the queue is available for transmission; a plurality offirst level shapers that are each associated with a subset of theplurality of queues, wherein each first level shaper: identifies apacket having a first level highest packet priority of all packetsavailable for transmission from the subset of the plurality of queues,and shares the first level highest packet priority with at least oneother component; a second level shaper that: determines that the packetprocessor should transmit a packet, receives at least one first levelhighest packet priority shared by at least one of the plurality of firstlevel shapers; identifies an available packet associated a highestpacket priority based on the at least one first level highest packetpriority, and transmits the available packet having the highest packetpriority; and a plurality of third level shapers between the pluralityof first level shapers and the second level shaper, wherein each thirdlevel shaper of the plurality of third level shapers comprises a rateshaper that: monitors an activity level for the third level shaper,determines whether the third level shaper is eligible to transmit apacket based on the activity level, and if the third level shaper isineligible to transmit a packet, indicates to the second level shaperthat no packet is available for transmission.
 23. The packet processorof claim 22, wherein the available packet associated with the highestpacket priority is selected from a group of packets identified aseligible for transmission by at least a first plurality of shapers. 24.The packet processor of claim 22, wherein each packet priority isdetermined based on at least one field carried by the associated packet.25. The packet processor of claim 22, wherein each queue of theplurality of queues comprises an activity monitor that determines astatus for the queue based on observed activity; and the packet priorityfor each packet is based on the status of the queue that stores thepacket.
 26. The packet processor of claim 25, wherein the status is oneof committed information rate status and excess information rate status.27. The packet processor of claim 22, further comprising: a third levelshaper associated with at least one first level shaper; wherein thefirst level shaper shares the first level highest packet priority withat least one of the third level shaper and the second level shaper.